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ABSTRACT 

A strategic planning model for planning and 
conducting outcomes assessment was developed and implemented at 
Wayland Baptist University in Lubbock, Texas, First, the Secretary’s 
Commission on Achieving Necessary Skills (SCANS) model was used to 
survey 100 undergraduate students regarding their perceptions of 
skills they would need after graduation. The SCANS survey findings 
were used during implementation of the following seven-step model 
planning process: select planners for the outcomes assessment; assess 
strengths, weaknesses, opportunities, and threats; conduct a thorough 
needs assessment and identify desired outcomes; establish 
administrative, methodological, feedback, and component strategies; 
design and/or identify appropriate instruments, strategies, and 
methods of data collection; gain acceptance for the assessment agenda 
from partners and internal/external constituencies; and gain 
commitment of resources and establish joint planning processes for 
the future. After analysis of the assessment findings, plans were 
formulated to develop the following; student orientation course to 
prepare students for Wayland* s degree program and for outcomes 
assessment; capstone course for all graduating seniors; and 
objectives and success criteria for the university and each of its 
units/divisions. The process of collecting and using outcomes 
assessment data to improve Wayland* s programs has already begun, 
(Appended are transparency masters detailing the model and the survey 
findings and outcomes assessment surveys/checklists,) (MN) 
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OUTCOMES ASSESSMENT: FROM KNOWLEDGE TO PRACTICE 
PRESENTATION AT THE AMERICAN VOCATIONAL ASSOCIATION, DENVER, 

COLORADO, DECEMBER 2, 1995 



BY KENT BROOKS, DIRECTOR 
WAYLAND BAPTIST UNIVERSITY-LUBBOCK CENTER 
LUBBOCK, TEXAS 



ASSESSMENT PLANNING MODELS 

The expanding interest in quality and accountability created a need for models from which to plan 
outcomes assessment. W. A. Shewhart of Bell Telephone Laboratories developed what may be 
considered the first model to problem-solving as it relates to quality— the PDCA (Plan, Do, Check. 

Act) Cycle. This model calls for planning to acconplish some goal, do the work, check effectiveness 
by gathering data, and acting to continue those things that were successful and refine those that were 
not. I have used this model to form a definition of Outcomes Assessment. Transparency 1. 

Through the subsequent work of Deming, Juran, Crosby and others, Shewhart's basic thoughts have 
evolved into today's concept of total quality management, or other such terms as total quality 
curriculum, and continuous quality improvement. 

"What Work Requires of Schools-A SCANS Report for America 2000 f commonly known as 
"SCANS" was released in 1991. The SCANS document reports skills and competencies, identified 
by the Secretary of Labor's Commission on Achieving Necessary Skills, needed for a high school 
graduate or anyone (including a college graduate) who is entering the workforce, and especially those 
who wish to be successful. 

The SCANS Report is divided into two categories. Category' One, Foundation Skills, consists of 
Basic Skills. Thinking Skills, and Personal Qualities Category Two, Competencies, consists of 
Resources. Inter personal Skills. Information. Systems, and Technolog y. It is recommended that 
SCANS be used as a tool to identify applicable school to work transition skiUs, and as a basis for 
defining outcomes required by our graduates. I used the SCANS model to survey 100 of our 
undergraduate students in 1994 to determine tneir perceptions of the skills needed after graduating 
from our university. The average age of these students was 36 years, and approximately 95 percent 
were already in the workforce. What they revealed is shown in the next three transparencies. 
Transparencies 2, 3. 4. 

Dr. Bobby HalL Director of Research and Institutional Effectiveness at Wayland Baptist University, 

^ has determined that a strategic planning model is needed for assessment at our university. His model 

is composed of seven phases, as shown in this illustration. Transparency 5. 
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Strategic Plaiuiing for Outcomes Assessment should impact 8 areas, as shown in transparency 6. 
These planning models have enabled us to have better use of previous assessnaent data, and to 
develop new assessment tools. 

Assessment leadership must begin with strong support of the chief academic officer, who must 
develop and sustain an environment conducive to outcomes assessment, according to Dr. Hall. Many 
committees and subcommittees are involved in the process of carrying the Wayland plan out to 
completion. The einpliases on outcomes assessment and quality control are both directed at making 
our university more accountable and productive. The total process is complex, and involves much 
analysis and creative thought. Dr. Bobby Hall is providing great leadership for developing and 
implementing our outcomes assessment plan, 

ASSESSMENT MEASURES 

Our accrediting agency, the Southern Association of Colleges and Schools, mandated o'-tcomes 
assessment two years ago. A comprehensive outcomes assessment program at Wayland Baptist 
University has been uixler development for a year. A portion of our efforts have been expended in 
reviewing various studies completed since 1989. Representative committees were formed in 1994 
and questionnaires for current students, graduating seniors, alumni, and community leaders were 
developed. The majority of fecuhy and staff members have been involved in this development effort. 
The instruments have been pilot tested with focus groups, and surveys of target groups have been 
completed. A complete list of opinion surveys to be conducted are shown in transparencies 7 and 
8. In addition, we have selected an ACT COMP (College Outcomes Measurement Program) test 
to measure the general education core, and administered this test to a sample of students early in 
1 995. This test is the most widely used measure of the academic foundation core in bachelor degree 
programs. We are using the objective test, which measures critical thinking, analytical, math, art 
appreciation, and other applied ktwwledge and skills. We will likely use this test for a period of time, 
and then develop our own exam to cover general education knowledge and skills. AH Wayland 
graduates are also required to complete the Graduate Record Exam. 

We must also determine how students who completee technical-vocational education programs from 
community colleges will be measured. Will the community college provide their documentation on 
outcomes assessment of the students' technical field, or must we re-evaluate them using national or 
locally prepared exams? 

We are also reviewing the Education Testing Service exams over major fields of study for possible 
use in our essessment program. 

Assessment also involves a measure of entering student skills. We will likely adopt the TASP exam 
which is the standard exam administered to all entering Freshmen at Texas Public colleges and 
universities. At present, we require the Nelson-Denny test for Reading and Comprehension, and a 
locally pre-algebra test. We will also emphasize writing across the curriculum with writing samples 
provided by teachers and scored by English faculty. 
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ACCOMPLISHING THE TASK OF ASSESSMENT 



A one semester-hour new student orientation course will be used to prepare students for their degree 
program at Wayland, and for outcomes assessment. This transparency shows what will be covered 
in the 15 classroom hour course. Transparency 9. Special emphasis will be provided on how to 
maintain a portfolio of student accomplishments during their program. This portfolio will include 
documentation of the mastery of SCANS skills. This portfolio will be extremely useful during the 
capstone course, in which studems will demonstrate their writing, speaking, interpersonal and other 
skills. Items to be included in the portfolio are shown in Transparency 10. 



Students must score 70 or above on an orientation test. To assist in passing this test, we have 
devetoped a computer-assisted self-study module, using the Microsoft Powerpoint slide show viewer. 

A two semester hour capstone course will be completed by each graduating senior. Much of the 
assessment data will be collected during this course. Transparency 1 1 shows what will be 
accomplished during the course. Again, students will demonstrate what they know and can do. 
Grades on coursework will not be enough evidence that graduates are prepared for career entrance 
or advancement. We will also collet data on SCANS competencies mastered by our students on the 
job and in the college/university curricula. 

As part of strategic planning, we are currently developing objectives and success criteria for the 
university and for each unit/division within the university. External evaluators will be invited to 
review our assessment data in relation to criteria for success. 

USING THE DATA 

We must show evidence that assessment data collected is used to change or improve programs. This 
is the most difficult task, but with great benefits. We have already begun to use the data collected, 
including upgrading the computer laboratory to meet the specific needs of the working adult student, 
revising the computer curriculum, developing the capability for our students to use Internet, and 
developing the capability for our faculty to use the latest computerized presentation systems. We are 
stressing computer competencies across the curriculum. Access to computer technology is a must. 



Tum-a-round time for student degree plans, grades, student accounts, and financial aid processing 
was too long. Quality is getting more than you expect, so we implemented a plan to have a response 
to a student's request within four hours and developed a new credo "whatever it takes." Students 
who are not getting the service they expect are encouraged to call the chief academic officer. 

Smdents wanted real coffee in the lounge, and we provided Folger's coffee! We added new signs 
to the building to enhance the appearance of our facility, which students said "were nice but 
somewhat cmde" for the progressive area of our city in which it is located. We changed our working 
hours to be in the office more in the evening to accommodate student needs for counseling, phone 
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rails paying accounts, grievances, and enrollment. Finally, we started giving students more attention 
in the classroom by visiting classes more often, aiid having them express their concerns for their 
program. One student told me that the "Hawthorn Eflfect" was much in evidence after our visits to 
tlte classes. I attribute much of the growth for our current Winter Term to special attention given in 
staff meetings to student service, and to addressing student concerns in the classes. 

We are contacting organizations where our students work, aixi offering to do counseling, registration 
and courses on site. Employers are enthusiastic about this kind of commitment to service. 

Assessment data can also be perplexing. A survey of new students who started with Wayland in 1 994 
revealed that our retention rate was less than 40 percent. We plan to contact these students to 
determine why they are no longer enrolled. 

EXAMPLE ASSESSMENT FORMS 

In your packet are several assessment forms we have developed and have begun to use. These 
include forms used for surveying current students, graouating seniors, and community leaders. I hope 
these may be of some assistance. If I could do this talk next year, we would have a much greater 
record of assessment activities. We intend to have answers for any question that our accrediting 
agency might ask during their visit with us in 1998. Wayland has over 3,000 students in extension 
centers in 5 states. The quality of the extension center programs must equal or exceed those provided 
at our main campus in Plainview, TX, 

SUMMARY 

I trust the information I have provided about assessment models, measures used to assess outcomes, 
and how to gather and use assessment data will be helpful to you. 
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A MODEL OF THE STRATEGIC PLANNING PROCESS 
FOR OUTCOMES ASSESSMENT 




(Selecting the 
Planners for 
Outcomes Assess- 
ment) 



(Assessing Strengths, 
Weaknesses, 
Opportunities and 
Threats) 



(Conducting a 
Thorough Needs 
Assessment and 
Identifying Desired 
Outcomes) 



(Establishing 
Administrative, 
Methodological, 
Feedback, and 
Component Strate- 
gies) 



(Designing and/or 
Identifying Appro- 
priate Instruments, 
Strategies and 
Methods for Data 
Collection) 



(Gaining Acceptance 
for the Assessment 
Agenda from 
Partners aud both 
Internal and Exter- 
nal Constituencies) 



(Gaining Commit- 
ment of Resources 
and Establishing 
Joint Planning 
Processes for the 
Future) 
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NEW STUDENT 
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ASSESSMENT/CAPSTONE COURSE 
■ ALUMNI PARTICIPATION 



PORTFOLIO INSERTS 
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■ TOOLS FOR SUCCESS 



CURRENT STUDENT QUESTIONNAIRE 



Please read instructions carefully before beginning each section. Completion time: 25 minutes. 

Definitions: Faculty - those who teach courses 

Staff - those who assist you in Wayiand offices, such as secretaries, clerks, etc. 
Administration - those who make policy and direct the institution, such as the president, 
vice presidents, office directors, etc, 

SECTION I - BACKGROUND INFORMATION 



1. Campus You Attend 2. Male Female 3. Civilian Military 

4. Classification: Freshman Sophomore Junior Senior Graduate Student Non-Degree Student 

5. Are you a transfer student? Yes No If yes, how many hours did you transfer to WBU? 

6. Employed? Yes No 7. Job type during academic year (check all that apply): 

On Campus Off-Campus Part-Time Full-Time 

8, Race: African-American Hispanic White Asian Native American 

Other (Please List) 

9. U.S. Citizen? Yes No 10. Age: 18-23 24-30 31-Above 11. Athletic team member? Yes No 



12. Do you receive financial aid? Yes No If yes, what percentage of your Scholarships 

living and college expenses Grants 

comes from: Loans 

On-Campus Job 

13. Do you live in: Wayiand dormitory Off-Campus Job 

Wayiand off-campus housing Parents 

Other Savings 



Total 100% 



14. Religious Preference: 

■ Southern Baptist Lutheran Episcopal 

Other Baptist Methodist Jewish 

Presbyterian Church of Christ Non-Denominational 

Catholic Disciples of Christ No Religious Preference 

Other (Please Specify) 

15. Marital Status: Single Married 16. Children? Yes No How Many? 

17. How many semester hours w ill you have completed at Wayiand at the end of the current semester? 

0-15 16-30 31-45 46-60 61-75 76-90 91-105 106-120 121-Above. 

18. What is your major(s)? minor(s)? 

19. What is your overall grade-point average (GPA)? 4.0 2.5-2.99 

3.5-3.99 2.0-2.49 

3.0-3.49 Below 1.99 



20. Why did you choose to attend Wayiand? (Check all that apply) 

Academic reputation Flexibility in class scheduling Flexibility in degree credits 

Christian environment Academic offerings in my field of study Friends/Family influence 

Location ^ Athletic programs Minister/Church influence 

Cost Financial aid or scholarships Small classes 

Size of school Social opportunities ‘ ^0 Other (Specify) 



Copyrighted. Wayiand jJaptist University. 1995 



SECTION II - PLANNING & ORGANIZATION 

Please rate each of the following items by checking the appropriate response: 


Excellent 


Very Good 


Adequate 


u. 

O 

0 

01 


Not Rated 


The preparation you are receiving for a career 












Academic counseling for your program of study 












Schedule of course offerings 












The quality of teaching relative to stated course objectives 












The relationship of courses & requirements to your overall degree objectives 












The quality and accuracy of printed information about the University (catalog, view book, etc.) 












Overall orientation for new students - finding what you need, who to see, where to go, etc. 












General liberal arts education 













What are your plans after yo>; graduate or leave Wayland? (check ail that apply) 

Enroll in a graduate program Employment is not a goal 

Continue in present job Transfer to another university 

Take a new Job Devote full time to home/family/personal activities 

Enter or stay in the military Other 

SECTION III - ACADEMIC EXPECTATIONS AND PRACTICES 



Please rate each of the following items on three scales. 

First, indicate how important you considered each item to be 
WHEN YOU ENROLLED tEntering Exnectation). 
Second., indicate how important YOU NOW CONSIDER 
t„ch item to be (Current Expectation), 

Third, indicate how well Wavland PROVIDES FOR EACH 
ITEM fCurrent Evaluation ). 


Entering j 

Expectation j 


1 Current j 
[ Expectation | 


1 Current 

1 Evaluation 


n 

>^ 

L. 

V 

> 


Somewtuil Important 


Not important 


No Opinion 


Very Important 


Somewhat Important 


^ 1 

£ [ 

Z 1 


Excellent 


Very Good 


Adequate 


Poor 


C 

rt 

o 
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The availability of library resources for courses taken 


























The accessibilitv of professors 


























The administration's caring concern for students 


























The size of classes 


























The quality of courses m preparing lor graduate scnooi 


























Hie avaiiabiliry ofaxtivities and ciubs at Waylana 


























I'he caring concern of office staff for students 


























Op jrtuniiies for student involvement in WBU decision maiting 


























Overall enrichment or your; ruing skills 


























speaking skills 


























:ruical thinking s\ills 


























.uuiyiicai sivilis 


























Jacer^^up 


























^■'.tristian iite commitmeiiL skills 


























seir-discipline 


























,;uerpersonal skiiis 


























library research SKiils 


























computer skills 


























Biblical knowledee 


























math skiiis 


























vOreer awareness 


























ciobal awareness 


























'Cientitlc awareness 


























proDlem-solving skills 


























,ippreciaiion or music lV an 
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aiitural awareness 
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Please rate each of the following items by checking the appropriate response: 

1. The amount your academic work requires the use of library resources? 
Too Much About Right Not Enough Not Rated 



2. Course grading? 

Too Hard About Right Too Easy 



3. Time required for study outside of class? (check two) 

Too Much About Right Not Enough More than time spent in class Less than time spent in class 

4. In your opinion, is crime a problem on the premises where you attend Wayland? (Plainview, San Antonio, etc.) 

A Major Problem Somewhat of a Problem Not a Problem Not Rated 

5. In your opinion, are alcohol and drugs used by WBU students? 

Used by Many Used by Some Not Used 

6. Are more fine arts/cultural activities such as concerts and films needed on campus? Yes No 



7. Are more religious activities needed on campus? Yes No 

8. Do you need child care services on campus? Yes No 

If yes, when is child care most needed? (check all that apply) Morning Afternoon Evening Saturday 



SECTION IV - COLLEGE ENVIRONMENT 

What is your current level of satisfaction with each 

1. Office of the Registrar 

2. Registration procedures 

3. Athletic programs 

4. Athletic facilities for student recreational use 

5. Student center 

6. Bookstore 

7. Classroom facilities 

8. Student housing 

9. Food services 

10. Religious/Spiritual environment 

1 1. Business Office 

12. Financial Aid Office 

13. Social/Recreational opportunities in Plainview 

14. The quality of personal counseling services 



of the following items? 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 

Excellent Very Good Adequate Poor Not 



Ratcd^ 
Rated_ 
Rated ^ 
Rated^ 
Rated^ 
Rated 
Rated_ 
Rated_ 
Rated 
Rated 
Rated 
Rated 
Rated 
Rated 



Please rate each of the following items on two scales. 

First, infiirate WHAT YOU EXPECTED when you enrolled 
at Wavland. 

Second, indicate vour CURRENT EVALUATION of that item 
at Wayland. 


Entering Expectation 


Current Evaluation 


Excellent 


Very Good 


Adequate 


o 


Not Rated 


Excellent 


Very Good 


Adequate 


; ^ 


Not Rated 


General attitude of students toward students 






















General attitude of students toward faculty 






















Cieneral attitude of students toward administration 






















General attitude of students toward staff 






















General attitude of faculty toward students 






















General attitude of administration toward students 






















General attitude of staff toward students 
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SECTION V - FINAL WORD 

1. How satisfied are you with the education you are receiving at Wayiand? 

Very Well Satisfied Satisfied Somewhat Dissatisfied Very Dissatisfied 

2. How satisfied are you with your overall experience at Wayiand? 

Very Well Satisfied Satisfied Somewhat Dissatisfied Very Dissatisfied 

3. Would you recommend Wayiand to others? Yes No 

Please write legibly! Use hack i f needed. | 

4. What things about Wayiand have you found to be most useful and helpful to you? Please be specific. 



5. What things about Wayiand are you least satisfied with? Please be specific. 



6. What specific suggestions do you have for improving Wayiand? 



Thank you for your help in improvinf^ IVa- land Baptist University. 
For additional information about the resu' j of this survey, contact the 
Office oj Instiiuiionai Research and Effectiveness, 

Box 5 "5 IVBU. or call 296-4523. 
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TENTATIVE 



WAYLAND BAPTIST UNtVERStlY 
TOTAL QUALITY CURfUCULUM 



Student Name;_ 
Cours»:_ 



Profile of Outcomes 

Bachelor of Science in Occupational Education 
SS#: 



Term and Date: 



INSTRUCTIONS: Check the outcomes that students acquired in above coui je. Indicate if students gained 
knowledge, skilis and/or pradicai experiences. 



OUTCOME AREAS 



OUTCOME LEVELS 



Resource Skills 


Knowledae 


Skills 


Rastisiti 

Exosoence 


1. Allocate Time 








2. Allocate Money 








3. Allocate Materials, Fadtities, and Resources 








4. Allocate Human Resources 








5. /kilocate Space 








6. Serve on a Committee 








7. Develop a Flow Chart 








8. Categorize Data 









loteroersoniLSJtlUs 








1. Participate as a Team Member 








2. Teach Others 








3. Serve Clients/Customers 








4. Exerctse Leadership 








5. Negotiate Agreements 








6. Follow Standard Business/Empioyment Practices 








7. Manage Personnel 


j 






8. Practice Christian Standards 








9. Communicate with Peers 








10. Train Others 








1 1 . Motivate Others 
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Knowlettoe 


Skills 


PTKtigai 


12. IdeiTtifY and Solve Probl«nis 








13. Mediate Problems 








14. Model Appropriate Behavior 








1 5. Serve on a CommunitY Organization 








16. Perform Church KSniitnes 








17. Apply Scripturea 








18. Make a Speech 








19. Supervise Personnel 








20. Develop Client Relations 








21. Intsrviewfora Job 








22. Market Self 








23. Counsel Others 








24. Plan and Conduct a Meeting 










InfbfmatigLSMte 








1. Acquire/Evaluate Infonnation 








2. Orgamze/Maintain Information 








3. Interpret and CommunicatB Information 








4. Conduct Library Researcn 








5. Write Reports 








6. Apply Algebra 








7. Apply Career Information and Data 








8. Apply Community Information and Data 








9. Apply State Information and Data 








1 0. Aopiy National Information and Data 








1 1 . Apply Global Information and Data 








1 3. Apply Scientific Information and Data 








14. Set and Accomplish Goals 1 







I 



I 15. PerTorm Creatjve Worx 
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Knowtadott 


Skilla 


Practical 

EzDinanca 


16. App*y Psychology 








17. Apply Economtcs 








1 8. Apply Soctai information and Data 








1 9. Apply Servfca Industry Informatksn and Data 








20. Write a Resume 








21. Articulate Meaning oftheBSOE Degree 








22. Participate in Self-Devetopmerrt Progranw 








23. Apply Coping Skiils 








24. Apply Stress Management Principles 








25. Apply Time Management Principles 








26. Apply Crisis Management Principles 






27. Apply Perseverance Principles 






28. Serve as a Resource Person 








29. Apply Management Principles 




I 

I 




30. Manage Money 
















System? Skills 








1 . Understand Systems 








2. Monitor Performance 








3. Correct Performance 








4. Improve Syster 3 








5. Design Systems 








6. Define Occupational Goals 








7. Survey Employment Opportunities 








8. Evaluate Career Goals | 








9. Develoo a Leadershio Project 






10. Create a Professional Portfolio | 






11. Conauct a Survey 







.Kpgwitdai 



TwHiwteg^lMte 

1 . SdkKt Technology 

2. Maintain and Troubt —faaot Technology 

3. Process Information by Computer 

4. Acces s Information and Data by Computer 

5. Use Commur^ic ation Equipmerrt 

S. Use Job-Specific Technology 



INSTRUCTOR SIGNATURE 



